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Abstract 

The purpose of this study was to describe West Virginia secondary vocational education 
teachers' use of student assessment information in making instructional decisions. A cluster 
sample of 240 teachers was needed for the study. A four-part questionnaire was designed to 
measure selected variables. Descriptive statistics were used to summarize the data. Teachers had 
an average of 15 years of teaching experience and nine years of related work experience. 
Attitudes toward assessment were viewed as "positive" by respondents. Secondary vocational 
education teachers neither agreed nor disagreed that they were constrained in their assessment 
activities. However, vocational education teachers must have opportunities during the school day 
to collaborate on the analysis of student work and to plan appropriate instructional 
improvements. 

The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 required 
accountability of all states that accept federal funds to support vocational programs. This was to 
be achieved through a system of specified performance measures and standards, which track 
both academic and occupational competency gains. Measures of performance were requested to 
address occupational competency attainment while measures of learning and competency gain 
were to reflect the achievement of basic and advanced academic skills. Goals 2000 and the 
School-to-Work Opportunities Act are two of the federal acts that emphasize high standards for 
all students while providing a framework and some financial incentives for public education to 
prepare all students for the world beyond school. 















Elegant arguments can be mounted regarding the probable effectiveness of the goals, testing, and 
standards initiatives. One may raise questions about institutional capacity, available resources, 
and the actual feasibility of the mandate type of policy approach. The more immediate concerns 
of many, however, are more technical than philosophical. Many measurement authorities, as 
well as school practitioners, are concerned about the ability of standardized measures commonly 
available to measure the desired outcomes with an acceptable degree of validity and precision 
(Asche, 1991). 

Statewide assessment cannot attempt to measure and thereby reflect all that local schools are 
able to achieve in terms of student outcomes. It has become necessary for schools to measure the 
attainment of their unique educational objectives (Perlman, 1991). The logical place for 
assessment activities measuring student achievement and perfonnance at the local level is in the 
classroom under the auspices of the classroom teacher. Wolf (1992) indicated that American 
education would become galvanized when new and more probing assessments are utilized to 
hold districts, teachers and students more accountable. 

The American Federation of Teachers, the National Council on Measurement in Education, and 
the National Education Association (1990) supported the concept of an instructional-assessment 
linkage by subscribing to the view that "student assessment is an essential part of teaching and 
that good teaching cannot exist without good student assessment" (p.l). Warmbrod (1993) drew 
attention to the linkage between instruction in agricultural education and assessment. He stated 
that "a dimension in the content of agricultural education that has roots in the critical analysis 
and systematic study of teaching is the assessment of learning and the evaluation of the 
effectiveness of educational programs" (p.7). 

Prior to the 1980s, the primary focus of research in assessment was not focused on the classroom 
but rather upon the area of standardized forms of assessment (Stiggins & Bridgeford, 1985). 
Classroom assessments, made by teachers have not been examined in depth. Stiggins and 
Conklin (1992)stated that 

"Although we were in an outstanding position to construct and administer high-quality, large- 
scale-testing programs in the early 1980s, we were far less able to teach teachers how to address 
the task demands of the day-to-day measurement of student achievement." (p. 11) 

A review of literature identified six general categories of classroom assessment methods as 
representing the various individual assessment instruments or procedures which teachers use to 
generate information for decision-making. The categories of assessment methods revealed were 
(1) objective paper and pencil items, (2) standardized test scores, (3) performance assessments, 
(4) informal observations, (5) essay-type items, and (6) portfolios. 

The educational measurement community is engaged in an equally serious rethinking of the 
structure of assessment (Wolf, 1992; Mehrens, 1992; Wiggins, 1989). Traditional, selected- 
response methods (multiple choice, matching, true-false) are being criticized for a variety of 
reasons: they can lead to narrowing of curriculum, test preparation practices may inflate scores 
in high stakes situations, and there are consistent differences in average performance between 
racial/ethnic and gender groups. (Koretz, Linn, Dunbar, & Shepard 1991; Shepard & Dougherty, 
1991; Smith & Rothenberg, 1991). 

Many educators believe that assessment can play an important role in systematic educational 
changes such as those being envisioned for vocational education (Wolf, 1992). However, there is 
a paucity of information in the literature about the assessment practices of vocational education 
teachers. How vocational education teachers use assessment information in the classroom and 
whether its use is effective can play a major role in enhancing and documenting both instruction 
and learning. 



Purpose and Objectives 

The purpose of this study was to describe West Virginia secondary vocational teachers' use of 
student assessment information in making instructional decisions. The specific objectives were 
as follows: 

1. To describe selected characteristics of secondary vocational education teachers in West 
Virginia: age, gender, related work experience, teaching experience, program area taught, 
education level, and certification route. 

2. To describe secondary vocational education teachers' perceptions of their use of student 
assessment data for making instructional decisions. 

3. To describe secondary vocational education teachers' attitudes toward the assessment 
process. 

4. To depict secondary vocational education teachers' perceptions of constraints in the 
assessment process. 

Method 

Research Design 

This descriptive study was designed to examine the use of assessment information, obtained 
from six types of student assessment methods, in addressing 10 instructional decisions. The 
dependent variable in this study was teacher use of student assessment information in 
instructional decision making. The variables, attitudes toward assessment and constraints to the 
assessment process were the independent variables of interest. 

Population and Sampling 

The target population (N = 647) was all teachers in West Virginia who taught full time at 
secondary vocational technical centers during the 1997- 98 school year. The 1997 - 98 West 
Virginia Education Directory was used to identify the population, which served as the sampling 
frame for the study. Using the formula recommended by Krejcie and Morgan (1970), a sample 
of 240 teachers was needed for the study. A cluster sampling technique was used to randomly 
select secondary vocational education teachers from the 32 secondary vocational technical 
centers in the state of West Virginia. Twelve secondary vocational technical centers were 
randomly selected to participate in the study in order to achieve the desired sample size of 240 
(12 schools with an average of 20 teachers per school). Secondary vocational technical centers 
were numbered from 1 to 32, and the 12 schools were selected using the random number 
generator in Microsoft Excel. According to Gay (1996), cluster sampling is more convenient 
when the population is very large or spread out over a wide geographic area. Sometimes it is the 
only feasible method of selecting a sample. It is not always possible, for example, to obtain or 
compile a list of all members of the population; thus in such cases, it is not possible to use 
simple random sampling. 

Instrumentation 

A four-part questionnaire was designed by the investigator for use in measuring selected 
variables. The dependent variable, teacher use of assessment information, was measured in part 
one of the instrument. The six types of assessment methods used in the study included: objective 
paper and pencil items, standardized test scores, performance assessments, informal 
observations, portfolios, and essay type items. 

Participants were asked to indicate the extent to which they use results, from each of the six 
assessment methods, in making 10 different types of instructional decisions. The 10 instructional 
decisions that were addressed were: planning instruction, diagnosing student weakness, 



monitoring student progress toward course objectives, communicating student achievement with 
parents, motivating students to leam, evaluating the effectiveness of instruction, evaluating the 
instructional materials used, grouping students for instructional activities, encouraging students 
to assess their own work, and assigning grades. A 5-point Likert scale ranging from "no use" to 
"considerable use" was used. 

Attitudes toward assessment were measured in part two of the questionnaire using a semantic 
differential scale. The scale was comprised of nine bi-polar adjectives, which described the 
concept "assessment. " A seven-point scale was used for each pair of adjectives. Participants 
were asked to use the scale for each adjective pair to describe their attitudes toward the overall 
assessment. 

In part three of questionnaire, nine statements were used to measure teachers’ perceptions of 
constraints to their assessment activities. Major constraints identified in the literature included: 
time, money, technology and assistance, training, autonomy in making assessmentrelated 
decision, and availability of assessment materials. The 5-point Likert scale used in this section 
ranged from strongly disagree to strongly agree. Participants were asked to indicate their level of 
agreement with each of the constraint statements. Teacher characteristics were listed in part four 
of the instrument. 

To ensure validity of the data, a panel of experts was used to establish content and face validity. 
The panel consisted of three vocational education teachers, a regional teacher educator, and two 
professors of vocational education. The instrument was field-tested for reliability with a sample 
of vocational education teachers (n = 11) not selected for participation in the study. Changes 
indicated by the validation panel and field test were made. Internal consistencies for the scales in 
the instrument were as follows (Cronbach's alpha): Use of Assessment Infonnation .94, Attitudes 
toward Assessment .95, and Constraints in Assessment .64, acceptable according to Nunnally 
and Beinstein (1994). 

Data Collection 

The Total Design Method (TDM) of conducting surveys (Dillman, 1978) was followed in all 
stages of the questionnaire construction and implementation process. A packet containing a 
cover letter, instructions for administering the questionnaire and copies of the questionnaire were 
mailed to the principal of each school selected for the study. A total of 240 questionnaires were 
sent to 12 principals during October of 1997. A total of 144 usable questionnaires were returned 
for a response rate of 60%. Because a number of questionnaires were returned uncompleted and 
there was no way to conduct appropriate follow-up procedures to control for non-response, the 
results of the study can only be generalized to the 144 teachers who provided usable data. 

Data Analysis 

Data were analyzed using the Statistical Package for the Social Sciences (SPSS Version 6.1 for 
Windows). Descriptive parameters, including frequencies, percentages, means, and standard 
deviations were used to organize and summarize the data. 

Results 

Demographic Characteristics 

The largest number of teacher respondents fell within the 42-51 year age bracket (52.1%). Of the 
144 cases, 56% of the respondents were male and 44% were female. Teachers had an average of 
15.02 years of teaching experience and 9.66 years of related work experience. Trade and 
industrial education teachers comprised the largest group in the sample and accounted for 43% 
of the cases. 



A graduate degree had been earned by 39% of the respondents. Respondents, who had 
completed a teacher preparation program on the job, and before receiving a bachelor degree, 
comprised 43% of the cases. 

Teacher's Use of Assessment Information 

Data in Table 1 depict teachers' uses of assessment information and educational decision 
making. Information generated from performance assessment was considered to be of more use 
to teachers in addressing educational decisions than any of the other assessment methods (M = 
4.19). Teachers rated performance assessment information to be of much use (M = 4.45) when 
specifically addressing the task of assigning grades. 

Teachers reported that information derived from informal observations (M =3.96) was of more 
use than all other assessment methods except performance assessment. Infonnal observations 
provided information that was considered by teachers to be of much use (M = 3.80 - 4.16) when 
addressing all 10 decision areas. 

Information obtained from objective paper and pencil items (M = 3.91) was found to be of less 
use than both performance assessments and informal observations, but of more use than essay 
items, portfolios, and standardized test scores. Information from this assessment method was 
found to be of much use (M = 3.68 - 4 .27) for nine of the decision areas. Teachers revealed that 
objective and paper pencil items were of some use when grouping students for instructional 
activities. 

Portfolios were found to be of only some use (M = 2.81) in providing information in the overall 
decision making process. However, teachers found portfolios to be of some use (M = 3.02) 
when specifically communicating student achievement results to parents. 

Standardized test scores (M = 2.75) and essay type information (M = 2.72) were the two 
assessment methods found to be of less importance to teachers when compared to the other 
methods. Teachers reported that standardized test scores were of some use when evaluating the 
effectiveness of their teaching methods (M = 3.03) and of limited use (M = 2.37) when 
assigning grades. For addressing educational decisions, information obtained through the 
implementation of essay type methods was found to be of some use for all ten decision areas (M 
= 2.50 - 2.84). 


Table 1 

Means 3 Standard Deviations for the Use of Assessment Information Generated From Six 
Assessment Methods (n=144) 


Decision 

Area 

Objective 

Item 

Standardized 
Test Score 

Performance 

Assessment 

Informal 

Observation 

Portfolio 

Essay 

Item 


M SD 

M SD 

M SD 

M SD 

M SD 

M SD 


4.00 1.01 

2.84 1.13 

4.24 .82 

4.09 .87 

2.68 1.17 

2.81 1.17 






































3.96 .99 

3.02 1.17 

4.20 .93 

4.16 .86 

2.73 1.24 

2.84 1.18 


4.13 .92 

2.72 1.19 

4.27 .89 

4.00 .95 

2.78 1.23 

2.81 1.10 


3.89 1.15 

2.83 1.27 

4.03 1.02 

3.80 1.09 

3.02 1.35 

2.72 1.20 


3.70 1.12 

2.62 1.10 

4.27 .83 

4.00 .97 

2.85 1.25 

2.61 1.09 


4.14 1.05 

3.03 1.29 

4.36 .78 

4.06 .90 

2.89 1.28 

2.82 1.30 


3.90 1.03 

2.85 1.20 

4.06 .91 

3.84 1.02 

2.79 1.21 

2.72 1.15 


3.49 1.18 

2.67 1.22 

3.93 1.10 

3.95 1.12 

2.62 1.24 

2.50 1.18 


3.68 1.23 

2.60 1.23 

4.16 .97 

3.80 1.13 

2.97 1.30 

2.60 1.27 


4.27 .97 

2.37 1.36 

4.45 .86 

3.97 1.03 

2.78 1.36 

2.77 1.27 

Overall 

Means 

3.91 

2.75 

4.19 

3.96 

2.81 

2.72 


Note : Educational Decision Areas: 


1. Plan for instruction 6. Evaluate instruction 


2. Diagnose student weakness 


7. Evaluate instructional materials 


3. Monitor student progress 


8. Group students 


4. Communicate student achievement 


9. Encourage self-assessment 








































5. Motivate students 


10. Assign grades. 


a Based on scale: 1= of no use; 2= of limited use; 3= of some use; 4= of much use; 5= of 
considerable use. 

Attitudes toward Assessment 

A semantic differential scale was used to measure teachers' attitudes toward the overall 
assessment process. Teachers were asked to respond to nine parts ofbi-polar adjectives, which 
were on either end of a 7-point scale. Figure 1 provides an overview of the distribution of mean 
teacher responses along a 7-point scale. The scale was as follows: 1 = extremely negative; 2 = 
very negative; 3 = negative; 4 = neutral; 5 = positive; 6 = very positive; 7 = extremely positive. 
Teachers reported that they perceived the assessment processes to be valuable, efficient, and 
important (M = 5 .77 - 5 .38). Overall, there was no indication that a negative attitude (M < 4 .0) 
existed towards the assessment process. 
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Figure 1. Distribution of Attitude Mean Scores 


Constraints to the Assessment Process 

Data in Table 2 describe responses to each of the constraint questions. Teachers tended to agree 
that they decided on what assessment methods to use in their courses (M = 4.28). Teachers also 
tended to agree that additional planning time would allow for assessment methods to be used 
more effectively (M = 3.94). Over two-thirds of the items were rated as "neutral" by respondents 
in this study (M = 2.69 - 3.00). 

Table 2 

Means 3 and Standard Deviations for Teacher Perceptions of Constraints to the Assessment 
Process (n = 144) 


Item 

M 

SD 

I decide what assessment methods to use in 
courses I teach 

4.28 

.97 

Additional planning time would allow me to use 
assessment methods more effectively 

3.94 

1.10 

Equipment is available in my school for use in 
scoring tests 

3.06 

1.41 

Quality published assessment materials are hard 
to find 

3.00 

1.18 

Funds are available for buying published 
assessment materials 

2.91 

1.32 

College courses were of little help in preparing 
me to assess student learning 

2.86 

1.25 


































































In-service activities have helped developed my 
assessment skills 

2.81 

1.15 

I have assistance in preparing student assessment 
activities 

2.75 

1.37 

I do not have information on published 
assessment materials 

2.69 

1.20 


Note. a Based on scale: 1 = strongly disagree; 2 = disagree; 3 = neither agree or disagree; 4 = 
agree; 5 = strongly agree. 

Discussion and Conclusions 

Based on the results of this study, the typical secondary vocational education teacher: 

1. is likely to be in the age bracket of 42-51 years old; 

2. completed an average of 15 years of teaching and 10 years of related work experience; 

3. completed a graduate degree; 

4. completed a teacher preparation program on the job; and 

5. is likely to be a teacher in trade and industrial education. 

Some of these findings are partially explained in a study reported by Lynch (1993). According to 
Lynch's study (1993), secondary vocational education teachers tend to have less formal 
education than others, but they have more related occupational experience and credentials. This 
emphasis on occupational experience in lieu of formal education is concentrated in trade and 
industrial education, where it has been guided by state policies in a tradition going back to back 
to the Smith-Hughes Act of 1917 (U.S.Congress, 1917). 

Performance assessments are particularly useful to secondary vocational education teachers. 
Given the fact that vocational education teachers use a competency-based curriculum, this 
finding revealed that performance assessment was of more use than the other five methods used 
in this study. This finding supports a study completed by Kershaw (1993). According to 
Kershaw's study, secondary vocational education teachers in Ohio were more likely to use 
performance assessments than any other assessment methods. 

Secondary vocational education teachers were more likely to use informal observations when 
grouping students for instructional activities. This finding suggests that when prompted, 
vocational education teachers can provide rich and detailed descriptions of their pupils. 
Secondary vocational education teachers placed much use on objective paper and pencil methods 
in assigning grades. This finding indicates that this method is well suited for assessing students' 
recall of factual knowledge. Portfolios were not found to be of much use for generating 
information in decision making. This was probably due to the fact that teachers have used 
different criteria for rating portfolio work or come up with different scores even when they use 
the same criteria (Borthwick, 1995). 

Secondary vocational education teachers were less likely to use standardized tests and essay type 















methods for assessing students than objective items, performance assessments, informal 
observations, and portfolios. This finding is supported by other studies (Green, 1990; Kershaw, 
1993) regarding the lack of standardized test scores in addressing educational decisions. This 
finding may also suggests that secondary vocational education teachers are not proficient in 
interpreting the use of information generated from standardized assessments, as well as how to 
construct and score essay questions for assessments. 

Attitudes toward assessment were viewed as being positive by secondary vocational education 
teachers. This suggests that vocational education teachers rely on the information generated by 
tests to provide them with the basis for improving instruction. In their review of literature, 
Scharfer and Lissitz (1987) concluded that although teachers may be ill trained to use accepted 
measurement practices, they see assessment as an important part of their professional role and 
have a positive attitude toward it. 

Secondary vocational education teachers neither agreed nor disagreed that they were constrained 
in their assessment activities. The results of this study mirror the findings of previous studies on 
constraints and assessment use (e.g., Gullickson, 1984; Kershaw, 1993). In addition to the 
predominantly neutral findings, both studies cite "limited time for planning" as a high level of 
agreement among respondents. Secondary vocational education teachers may therefore have less 
motivation to use the data if there is lack of time to address problems related to assessment 
quality. 

Implications and Recommendations 

There are a variety of ways for students to discover their workplace-related skills and 
knowledge with the help of educators. It is important to understand, however, that no single 
assessment method can completely measure a student's range of skills and knowledge in a 
content area. Thus, it is necessary to use several types of assessment methods to help students 
learn about their skills in even a single content area. Both written, short-answer forms of testing 
and diverse methods of performance assessment are likely to be used by vocational educators. 
However, vocational educators rely much more on diverse methods of performance assessment. 

So far there are no clear, unambiguous rubrics for evaluating each aspect of a portfolio. States 
that have begun to use portfolios on a large scale have had difficulty achieving acceptable 
quality in their scoring (Stecher & Herman, 1997). However, one approach is to establish 
guidelines for the contents of the portfolios so that they all contain similar components. Specific 
learner outcomes can be identified for each component and then techniques can be developed 
for assessing student perfonnance in terms of these outcomes. 

Based on the findings of this study, it is recommended that pre-service and in-service providers 
place more emphasis upon essay construction and scoring. They should also place more 
emphasis on interpreting the use of information generated from standardized test scores. 

Teachers must have opportunities during the school day to collaborate on the analysis of student 
work and to plan appropriate instructional improvements. School administrators can convey the 
importance of this work by providing time for assessment as an integral part of teachers' 
responsibilities. 

One of the main criticisms leveled against traditional assessments is that they are used to sort 
students and, on that basis, to deny educational opportunities (Darling - Hammond 1991). The 
consequence of instructionally sound assessment is quite different — it enhances the opportunity 
to learn. The assessment data are not used to label students. They simply provide information on 
areas in which students already do well, and focus on what they need to learn. Vocational 
teachers can use the results of such assessments to determine appropriate learning experiences 
and to guide the redesign of school programs and structures so that teacher and student 



performance improves. Obviously, trends in assessment will impact workforce education 
practice. For this reason, vocational educators need to take an active role in debates regarding 
assessment techniques and criteria. 

Educators and employers believe that the work world is changing and vocational education must 
adapt if it is to serve students well. The changes in the workplace are complex and not 
completely understood, but most observers believe that future employees will need integrated 
academic and vocational knowledge, a broad understanding of occupational areas, the ability to 
interact creatively with their peers, and higher-order cognitive skills that allow them to be 
flexible, learn rapidly, and adapt to ever-changing circumstances. To the extent this belief is 
true, vocational training needs to place greater emphasis on integrated learning, critical thinking- 
skills, and connections between vocational and academic skills, rather than on the mastery of the 
narrow, occupation-specific skills that characterized vocational education in the past. This new 
vision may also require broader changes in vocational education, including rethinking the 
organization, goals, content, and delivery of services, as well as the manner in which students 
and programs are assessed. 
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